Neurobiology of Language

Understanding the cognitive architecture of the human language faculty requires exploring the boundaries of both predisposition and environmental experience. However, previous research on extraordinary multilingualism has often confounded language aptitude with multilingual experience, obscuring their distinct neural correlates. Here, we leveraged a linguistically diverse sample (N=121) and extensive behavioural testing to dissociate language aptitude from multilingual experience, modelling both dimensions continuously in whole-brain speech processing. Language aptitude and multilingual experience were weakly related, and their dissociation was also evident at the neural level. Higher language aptitude showed a neural signature of efficiency, characterised by lower activation in core perisylvian regions. In contrast, higher multilingualism was associated with greater engagement of regions implicated in narrative, multimodal, and memory processing, and with recruitment of traditional language hubs only during degraded speech processing, likely reflecting active attempts to decode unintelligible input. Finally, aptitude and experience interacted within sensorimotor regions. Continuous quantification of multilingual experience proved more sensitive than artificial grouping. By disentangling language aptitude from multilingual experience, this work provides a more precise account of the multilingual brain, and shows that its neurobiology can be better understood by modelling predisposition and experience as distinct but interacting dimensions.

Aphasia is characterized by impaired word retrieval, yet most cognitive models of word production assume that underlying conceptual-semantic representations are largely preserved. This study investigated whether concept-level semantic structure remains decodable from BOLD signals in chronic post-stroke aphasia and which semantic models best explain neural representational geometry during covert semantic feature generation. Eight healthy adults and six individuals with chronic aphasia completed a dense-sampling fMRI protocol in which they viewed 57 pictured nouns while silently generating semantic features. Representational similarity analysis showed that an experiential model (Exp48) best matched neural geometry in both people with aphasia and controls, outperforming taxonomic (WordNet) and distributional (Word2Vec, GloVe) models. Using representational similarity decoding, concept identity was recovered well above chance in both groups. No relationship was found between decoding accuracy and language measures from individuals with aphasia. These findings suggest that experiential semantic structure remains robustly represented and decodable in chronic aphasia despite lesion-related language impairments, highlighting preserved conceptual representations alongside altered anatomy.

Irony and sarcasm are complex forms of non-literal language that hinge on a misalignment between surface meaning and speaker intent, requiring listeners to integrate contextual, semantic, and prosodic cues. While prior neuroimaging studies have implicated a broad network--including the temporal cortex, the inferior frontal gyrus, and the medial prefrontal cortex--in the comprehension of ironic and sarcastic speech, the precise neural mechanisms underlying the integration of semantic and prosodic information remain unclear. In the present study, we addressed this gap by employing voxel-wise encoding models to systematically identify brain regions specifically involved in combining prosodic and semantic cues during non-literal language comprehension. Participants listened to naturalistic auditory dialogues in which both discourse context and target utterance semantics and prosody were systematically manipulated. We derived custom text embeddings using transformer-based models to capture context-sensitive semantic representations of ironic statements, alongside acoustic features characterizing affective prosody. Ridge regression models were fitted to predict BOLD responses at the voxel level using semantic, prosodic, and combined features, and we identified integration as voxels in which each modality contributed predictive information beyond the other, using a permutation-based conjunction test. The regions integrating prosody and semantics depended on whether discourse context was modeled: integration was confined to the bilateral temporal speech cortex when statements were encoded in isolation, but additionally engaged the left inferior frontal gyrus pars orbitalis (IFGorb) when each statement was weighted by its relevance to the preceding context. These findings indicate that the left IFGorb integrates prosody with context-dependent meaning, engaging beyond the temporal speech cortex specifically when comprehension requires combining semantic, prosodic, and contextual cues--as in irony and sarcasm.

Second language (L2) processing engages both language-specific and domain-general control systems, yet how these systems vary with L2 proficiency remains unclear. We used functional magnetic resonance imaging (fMRI) to examine neural activity during L2 English processing in Korean-English (K-E) bilinguals across three proficiency levels (beginner, intermediate, advanced). Participants performed rhyme and spelling judgement tasks manipulating orthographic-phonological conflict. Behaviourally, conflict conditions reduced accuracy, with proficiency effects observed selectively in the rhyme task. fMRI results showed that conflict processing recruited frontoparietal control regions, including inferior frontal and parietal cortices, accompanied by deactivation in default mode network regions. Critically, proficiency-related effects differed by task. During rhyme judgement, advanced bilinguals showed greater activation in the left supramarginal gyrus (SMG) and cerebellum, whereas intermediate bilinguals exhibited greater recruitment of the left middle orbital gyrus and dorsomedial prefrontal cortex. During spelling judgement, advanced bilinguals showed greater thalamic activation alongside greater deactivation of the right dorsolateral prefrontal cortex. Activity in the left SMG and cerebellum was positively associated with L2 reading score, and cerebellar activity was also associated with rhyme-task performance, whereas right DLPFC activity was negatively associated with the scores. These findings suggest that increasing L2 proficiency is associated less with altered recruitment of core reading regions than with task-specific shifts in the balance between phonological-specialized, subcortical attentional, and domain-general control systems supporting L2 processing.

Fluent sentence production relies on the precise and timely coordination of lexical retrieval, morphosyntactic planning, and motor execution. While a left-dominant fronto-parieto-temporal network has been identified for these subcomponents of sentence production, less is known about the time course of engagement of these regions. One key issue is the extent to which the engagement of neural regions is sequential. The aim of the present study was to delineate spatiotemporal patterns of subcomponents of sentence production, namely lexical retrieval, inflection, and constituent assembly, and identify any lexical category differences (nouns versus verbs) in these processes. Magnetoencephalographic (MEG) responses were measured using a novel picture-icon paradigm in which verbal responses were elicited using object or action pictures followed by an icon that indicated the subcomponent sentence processes: lexical access, inflection (plurals for nouns, tense for verbs), and constituent assembly (adjective + object or pronoun + action). MEG data from twenty native English-speaking right-handed neurotypical adults was analysed for both stimulus-locked and response-locked timelines. We focused on responses in left hemisphere speech-language regions of interest (ROI): precentral gyrus (M1), inferior frontal gyrus (IFG), anterior temporal lobe (ATL), posterior temporal lobe (PTL), and inferior parietal lobe (IPL). The results revealed the following: 1) early and largely simultaneous engagement of all ROIs for lexical access; 2) compared to noun lexical access, verb lexical access is associated with a larger neural response across multiple language ROIs; 3) morphosyntactic planning engaged all ROIs, showing commonalities as well as unique patterns for each morphosyntactic condition; 4) noun morphosyntax vs noun lexical access showed a larger response across multiple ROIs compared to verb morphosyntax vs verb lexical access; 5) M1 was coactivated with other ROIs early in the planning stage, suggesting simultaneity of language and motor planning. The findings support a network model of language in which the syntactic, semantic, and phonological properties of a word are bound to a single anatomically distributed representation.

Human language processing can be studied through both behavior and brain activity, yet it remains unclear whether these two data types reflect sensitivity to the same information. One influential view holds that both behavioral and neural responses are largely determined by processing effort, often estimated by word surprisal together with the context-independent properties of word frequency and length. At the same time, neural responses have been shown to encode richer aspects of linguistic content, including meaning. Here, we use neural network language models to operationalize these alternatives and systematically compare, within the same analytic computational framework, the predictive power of low-dimensional effort-based predictors and high-dimensional embedding representations that encode contextualized linguistic content, including meaning. Across 8 behavioral datasets and 5 neural datasets (4 fMRI and 1 ERP), we find that processing effort captures substantial variance in both behavioral and neural measures of language processing, in line with much previous work. However, for brain responses---but not for behavioral measures---embedding representations carry substantial predictive power beyond the estimates of processing effort. These results therefore suggest that neural data provide access to rich, high-dimensional dynamics of language comprehension, whereas behavioral data reflect a bottlenecking of these dynamics into a small set of theoretically motivated properties of contextualized linguistic input.

Language comprehension requires the continuous transformation of speech into a hierarchy of linguistic units, from phonemes to syllables to words. Because speech unfolds rapidly, listeners are thought to predict upcoming content to keep pace. Previous research has provided empirical evidence for predictive processes operating at multiple linguistic levels during naturalistic listening, including words and phonemes. However, it remains unclear whether prediction also operates concurrently at other levels, such as syllabic and phrasal representation. Here we use Mandarin Chinese to examine the neural signatures of predictive processing across multiple levels of linguistic granularity during natural speech comprehension. Mandarin comprises four representational levels: phoneme, sub-syllabic, character and word, and its lexical identity is largely constrained at the sub-syllabic level, potentially redistributing predictive weight across linguistic representations. We recorded magnetoencephalography (MEG) data while 34 native Mandarin speakers (21 females) listened to a naturalistic audiobook and applied linear regression modeling to examine how linguistic features modulated neural activity. We found that the brain activity of listeners segmented speech into hierarchical units, and that surprisal modulated responses simultaneously across sub-syllabic, character and word levels. In contrast to findings from Indo-European languages, however, we did not observe unique surprisal effects at the lowest, phonemic level. Furthermore, the surprisal of lexical tone in Mandarin modulated brain activity only when integrated with its phonological components. These findings suggest that predictive processing during Mandarin speech comprehension operates concurrently across multiple (though not necessarily all) levels of linguistic granularity, with its implementation shaped by language-specific structural properties. Significance statementLanguage comprehension involves segmenting a continuous acoustic stream into multiple linguistic units, from phonemes to words, and generating predictions at these levels. However, direct neural evidence remains limited regarding how segmentation and prediction operate simultaneously across levels of linguistic granularity, particularly outside Indo-European languages. Using temporal response function analysis of magnetoencephalography data recorded during naturalistic Mandarin listening, we show that predictive processing occurs across multiple levels of linguistic granularity. Specifically, we find evidence for prediction-related neural responses at sub-syllabic, character, and word levels, but not a reliable unique effect at the phonemic level. These results indicate that predictive processing also operates during Mandarin speech comprehension, and its neural implementation is shaped by language-specific structural properties.

Communicative abilities acquired over the first year of life are foundational for language development. Understanding the neural mechanisms underlying these abilities can explain individual differences and improve predictions of later language outcomes. While prior research on neural markers of language development has focused predominantly on oscillatory neural activity, the underlying aperiodic component of the neural signal may provide further insights into cortical maturation by indirectly reflecting variation in neural excitation/inhibition (E/I) balance. In neurodevelopmental conditions, both reduced and elevated aperiodic activity are linked to atypical development, and initial evidence from at-risk cohorts also relates aperiodic activity to language outcome. Moving beyond clinical populations, the present study examined whether aperiodic activity during neonatal resting-state EEG relates to 12-month communicative abilities in typically developing infants. Aperiodic offset and exponent were extracted from resting-state EEG recorded two weeks after birth in n = 50 neonates using the FOOOF algorithm. At 12 months, communicative abilities were assessed with the parent-report CSBS-DP Infant-Toddler Checklist. Quadratic regression analyses showed an inverted U-shaped relationship between the neonatal aperiodic exponent and communicative abilities: both lower and higher exponent values were associated with poorer outcomes, with intermediate values linked to better abilities. No robust association was found for the offset. These findings suggest that aperiodic activity captures variability in early communicative development, with an optimum at intermediate levels, consistent with a balanced E/I state. The scope of aperiodic brain activity as a developmental marker may thus extend beyond clinical populations to predicting communicative development in healthy infants.

The Visual Word Form Area (VWFA), located in ventral occipitotemporal cortex, plays a critical role in skilled reading. Researchers have theorized that the VWFA develops in its specific anatomical location due to the convergence of major white matter tracts and proximity to functionally similar regions. This suggests that precise anatomical positioning may be crucial for optimal VWFA function. Previous research has identified several functional differences in this region between typical and struggling readers (i.e. dyslexia): struggling readers show weaker text-selective responses and often exhibit a smaller or even absent VWFA. However, it remains unexplored whether the precise anatomical location of this region also differs between typical and struggling readers. We tested whether VWFA anatomy differs between children with and without dyslexia (N=87). Participants completed a functional localizer, which we used to manually define the VWFA in each individuals native anatomy. We examined whether: (1) VWFA anatomical location relates to reading ability, (2) children with dyslexia show greater variability in VWFA location compared to typical readers, and (3) VWFA location with respect to white matter tracts relates to reading ability. Results reveal that, despite being smaller in children with dyslexia, there is no relationship between VWFA location and reading ability. Specifically, individual VWFA location relative to anatomy, relative to others VWFAs, and relative to white matter tracts, is not related to reading ability. These findings suggest that while the VWFAs general anatomy may be facilitated by development, its precise location remains stable and unrelated to reading proficiency.

Understanding how native speakers and second language (L2) learners process language differently is fundamental to theories of language acquisition, comprehension, and use. A central debate in this literature concerns the inconsistent findings in emotional prosody processing between native speakers and L2 learners, partly attributable to stimulus lexicality: some studies use real words, whereas others use pseudowords. To resolve this discrepancy, we measured fMRI responses in 15 native speakers and 15 L2 learners while they listened to Mandarin real-word and pseudoword stimuli. Both groups engaged in overlapping auditory, cognitive, and affective regions, including the auditory cortex, inferior frontal cortex, superior temporal cortex, and the temporoparietal junction. However, the two groups showed distinct responses depending on lexicality: native speakers showed stronger recruitment of temporal regions for pseudowords, whereas L2 learners showed greater frontoparietal activation for real words. These patterns were consistent across positive and negative prosody conditions. Furthermore, among L2 learners, higher proficiency was associated with greater frontal activation for real words, but not pseudowords. Together, these findings demonstrate that the differences between native speakers and L2 learners depend critically on stimulus lexicality, highlighting the role of lexicality in bilingual emotion processing.

How internal speech relates to overt speech remains a fundamental question in speech production: do different forms of speech preserve a common neural representation of the intended word, or does that representation change as speech becomes articulated? We used time-resolved electroencephalography to characterize representations of 10 Chinese words during imagined, silent, and overt speech. Word identity was reliably decodable in all three modes, but its temporal dynamics differed: imagined-speech representations peaked earlier and were less temporally stable, whereas silent and overt speech showed stronger and more sustained representations. Cross-mode decoding revealed word-discriminative information shared across all three mode pairs, with substantially stronger generalization between silent and overt speech. However, direct comparison of word-level representational geometry revealed robust correspondence only between silent and overt speech, indicating that transferable information across modes does not necessarily imply preservation of the broader relational structure among words. Representational similarity analyses further showed distinct visual-form, semantic, and phonetic dynamics across speech modes, with late visual-form and phonetic information contributing uniquely to the geometry shared by silent and overt speech. Controlling for time-matched surface electromyography preserved the overall silent-overt neural correspondence and within-mode phonetic representations, while eliminating the unique phonetic contribution to their shared geometry, suggesting that peripheral articulation accounts for part, but not all, of this common structure. Together, these findings show that imagined, silent, and overt speech share word representations at different levels and suggest that representational geometry and temporal stability are progressively reorganized as internal speech is translated into articulation.

Syntax processing impairment is a defining feature of primary progressive aphasia (PPA), particularly in the nonfluent/agrammatic variant. However, current models are derived largely from Indo-European languages, and it remains unclear whether the cognitive and neuroanatomical mechanisms of syntax processing generalize to Chinese, an analytic, pro-drop language that encodes grammatical relationships primarily through word order and function words rather than inflectional morphology. Participants were recruited from the international Chinese Language Assessment in PPA cohort and included 64 individuals with PPA (19 logopenic, 15 nonfluent/agrammatic, 15 semantic, and 15 unclassifiable/mixed) and 60 cognitively healthy controls. Participants completed sentence comprehension and production tasks with systematically manipulated syntactic features. Item-level linguistic determinants were examined using ridge regression, diagnostic performance using receiver operating characteristic analyses, and neuroanatomical correlates using voxel-based morphometry of structural MRI. Performance on sentence comprehension and production was influenced by grammatical and lexical features. Subject-relative clauses, passive constructions, a higher proportion of function words, longer character length, and greater lexical content were associated with poorer performance among participants with PPA. Sentence comprehension was impaired in the logopenic, nonfluent/agrammatic, and semantic variants, even when isolated to the syntactic component of the sentence comprehension task. Similarly, sentence production was impaired across all PPA variants (all P < 0.01 versus controls). No significant differences were observed among PPA variants. Adjusted receiver operating characteristic analyses demonstrated high discrimination between each PPA variant and controls, with areas under the curve ranging from 0.87 to 0.97 for sentence comprehension-syntax and from 0.91 to 0.97 for sentence production. However, pairwise comparisons showed no significant differences in discriminative performance among PPA variants after correction for multiple comparisons. Both sentence comprehension-syntax and sentence production showed associations with semantic and repetition measures in addition to motor speech measures. Voxel-based morphometry showed that higher sentence comprehension-syntax scores were associated with greater grey matter volume in left frontotemporal and cingulate regions, whereas higher sentence production scores were additionally associated with greater gray matter volume in frontostriatal structures, including the caudate, putamen, and thalamus. Syntactic processing impairment in Chinese-speaking individuals with PPA extends beyond the nonfluent/agrammatic variant and is associated with distributed frontotemporal and frontostriatal regions, as well as lexical-semantic and auditory verbal short term memory mechanisms. These findings suggest that the cognitive and neural manifestations of syntactic impairment may be shaped by language-specific typological properties, underscoring the need for linguistically informed approaches to the assessment and interpretation of PPA.

Language comprehension involves the integration of single words (lexical units) into phrases and sentences (multi-word structures). Previous frequency-tagging studies have found that low-frequency neural responses synchronize to the frequency of multi-word structures. However, it is currently unclear how exactly structural and lexical processes jointly impact these synchronization findings. The present magnetoencephalography experiment implemented the frequency-tagging paradigm in the visual modality with written words to investigate neural synchronization to multi-word sentences varying in internal structure (reversed word orders between verb-initial Spanish and verb-final Basque sentences) and in lexical content (real words and pseudo words). We find converging evidence that neural responses largely synchronize to structural rather than lexical features. This was observed as robust phase synchronization strength to the frequency of sentences containing reversed structures, with certain lexical modulations depending on language-specific structural features. Crucially, we also found shifted phase angle dynamics between the reversed structures of Spanish and Basque sentences independently of word-level lexical characteristics. Together, these findings suggest that neural synchronization to multi-word structures is largely driven by distinct structural features operating via two segregated neural dimensions: frequency coding for the coarser aspects (i.e., timescale/duration) and phase representing the finer-grained aspects (i.e., internal structure) of multi-word structures. Our findings thus advance key insights into the core components of the neural mechanisms supporting language comprehension. HighlightsO_LINeural synchronization to sentences is driven by structural (not lexical) features. C_LIO_LIRobust sentence-frequency synchronization across languages varying in structure. C_LIO_LIPhase angle is selectively sensitive to cross-linguistic structural differences. C_LIO_LILexical modulations depend on language-specific structure. C_LIO_LIStructure synchronization segregates into two dimensions: frequency and phase. C_LI

Language can be experienced in highly serial form, as in speech, or in fully parallel form, as in text, yet how our brains adapt to differences in seriality is not understood. Here, we held modality constant (visual) and examined whether processing mechanisms underlying grammaticality detection are similar under Rapid Serial Visual Presentation (RSVP) and Rapid Parallel Visual Presentation (RPVP). To identify shared neural representations of grammaticality, we trained a classifier on magnetoencephalography (MEG) responses to grammatical and ungrammatical three-word phrases in one presentation mode and tested its generalization to the other. Behaviorally, accuracy was higher and reaction times faster for grammatical vs. ungrammatical phrases across presentation modes. Neurally, cross-presentation decoding revealed shared representations of grammaticality at word 2 across train-test directions--albeit with different latencies and durations--and an early shared representation at word 3 when trained on RPVP. We take the bidirectional representations at word 2 to capture distinct but presentation-invariant combinatorial processes and the asymmetry at word 3 as reflecting stronger global structure encoding in RPVP.

Can grammatical gender in a bilingual's first language shape gender-stereotype processing in a second language? Spanish (L1)-English (L2) bilinguals (n = 28) and English monolinguals (n = 28) completed an event-related potential (ERP) priming task in which English pronouns (SHE/HE) followed gender-stereotyped English nouns, half of which had gender-marked Spanish translation equivalents (e.g., NURSE 'enfermera/o', SURGEON 'cirujana/o'), and half unmarked translation equivalents (e.g., SINGER 'cantante', JANITOR 'conserje'). Both groups showed asymmetric stereotype priming: male pronouns elicited a larger N400 for incongruent than congruent primes, whereas female pronouns elicited a larger P300 for incongruent than congruent primes. Crucially, only bilinguals showed modulation by Spanish grammatical gender marking: the N400 effect for male pronouns was larger for primes with gender-marked than unmarked Spanish translations. These findings provide neural evidence that grammatical gender in a bilingual's first language can influence gender-stereotype processing in a second language, linking cross-linguistic activation to social cognition.

Developmental dyslexia is associated with impaired cortical tracking of continuous speech and difficulties in perceiving within-word prosodic structure in children and adults. For adults and continuous speech, both low-frequency delta- and theta-band impairments and high-frequency gamma- and beta-band differences are found. To date, perceptual difficulties with intra-word prosody have not been studied in direct relation to neural speech encoding. Here neurotypical adults and adults with dyslexia listened to multisyllabic words. EEG was recorded, and cortical encoding was quantified using temporal response function (TRF) modelling. Four speech representations were explored: lexical stress pattern (intra-word prosodic prominence), the low-frequency speech envelope, word onsets and syllable onsets. Adults with dyslexia showed significantly reduced encoding of intra-word prosodic prominence (delta band) and of the speech envelope (delta and theta bands). In contrast, encoding of word and syllable onsets was preserved. Pinpointing delta-band cortical tracking as central to encoding intra-word prosodic prominence opens new avenues for research.

Humans routinely encounter speech in languages they have never heard, yet how the brain responds to such input and whether bilingual experience shapes this response remains unknown. Here, we used electroencephalography (EEG) and temporal response function (TRF) modeling to examine cortical tracking of the speech envelope in 24 English-monolingual and 24 English-Mandarin bilingual adults. Participants listened to naturally produced continuous speech in three languages: English (familiar to all), Mandarin (familiar to bilinguals only), and Vietnamese (unfamiliar to all). We report two main findings. First, both monolinguals and bilinguals showed enhanced cortical tracking for unfamiliar relative to familiar languages, evidenced by higher EEG prediction accuracy (PA). Monolinguals showed enhanced tracking for both Mandarin and Vietnamese, whereas bilinguals showed enhancement only for Vietnamese, consistent with Mandarin being a familiar language for this group. This finding suggests that enhanced cortical encoding of unfamiliar speech is a general property of the listening brain, not a signature of listening to a non-native language or reduced language proficiency. Second, bilinguals strikingly showed stronger cortical tracking than monolinguals overall, in both PA and TRF peak weights, with the TRF peak weight advantage present across all three languages, suggesting a difference in how bilingual experience shapes the neural encoding of speech. These findings have implications for understanding how the brain navigates the linguistic diversity of everyday life in an increasingly global, multilingual world.

Comprehending spoken language requires extracting syntactic structure and meaning from a continuous acoustic signal. Psycholinguistic evidence points to a key role for prosody in this process. However, because interactions between prosody and syntax are typically demonstrated using carefully controlled sentence constructions, it remains unclear whether such interactions constitute a general, default property of spoken language processing. Addressing this question requires naturalistic, broad-coverage data. Here, we combined experimental psycholinguistics with naturalistic neuroscience to investigate the joint neural encoding of syntax and prosody in MEG recordings of 26 participants (19 female) listening to narrative speech. Linguistic annotations revealed that strong prosodic boundaries tend to coincide with syntactic phrase closures, and that the strength of prosodic boundaries scales with the number of phrases closed. Using time-resolved regression with temporal response functions, we modeled how these features relate to delta-band neural activity during continuous speech processing. Our results reveal a bidirectional interaction between syntax and prosody. Prosodic marking modulates the neural processing of syntax, with phrase closures being encoded more strongly when accompanied by strong prosodic boundaries. Conversely, syntactic structure shapes prosodic encoding, such that boundary strength is neurally encoded primarily at points of syntactic phrase closure. These findings demonstrate that syntax and prosody are encoded not independently, but in a context-dependent and mutually constraining manner, highlighting their joint role in guiding structure building during naturalistic spoken language comprehension.

Understanding speech requires listeners to integrate incoming input with prior linguistic and thematic knowledge to access meaning, a task greatly aided by prediction. Surprisal and related phenomena (e.g., next word prediction) tend to be associated with broad activation of language regions during listening. A major challenge for interpreting the neural correlates of surprisal is that lexical expectancies are constrained both by syntactic and semantic information. In contrast, semantic distance yields more of a pure metric of relatedness between one language constituent (e.g., word, phrase, n-gram) and another within a high-dimensional semantic space. We contrasted surprisal and semantic distance in running discourse during naturalistic audiovisual language comprehension. We analyzed fMRI scans from 20 English-speaking adults who viewed the full-length film 500 Days of Summer. Our focus was on brain sensitivity to either surprisal or semantic distance as language unfolded word-by-word. To this end, we implemented a series of hierarchical voxelwise encoding models. After accounting for low-level auditory and visual features, the addition of either surprisal or semantic distance improved prediction in bilateral superior temporal gyrus (STG), left inferior frontal gyrus, and right cerebellum. Critically, both semantic distance and surprisal explained unique variation in bilateral temporal cortex after accounting for the other. These results support the notion that the brain tracks both broad probabilistic prediction as well as semantic integration. More generally, these findings extend previous work on auditory-only speech by demonstrating that surprisal and semantic distance predict meaningful neural variation even within a rich audiovisual context.

Researchers in artificial intelligence increasingly intervene on language models to study how their functions are organized, silencing weights and attention components in ways reminiscent of the brain lesions long used to map language in post-stroke aphasia. Under this program of mechanistic interpretability, a common move is to ablate an attention head and read the resulting drop in a behavior as evidence that the head implements it. How much that move can establish about where a behavior is computed remains unclear, because a head whose removal disrupts a behavior is necessary for it but need not be the site where it is computed. We examined this question for the picture naming task, taking the loss of naming (anomia) that defines aphasia as the behavior of interest. Across six vision-language models, spanning three language backbones and a range of parameter scales, we ablated each attention head in turn during single-word picture naming and measured accuracy before and after. The degree of localization varied widely across models. In LLaVA-1.6-Vicuna-13B, a single early head (layer 0, head 20) was necessary: removing it alone reduced naming accuracy from 0.99 to 0.006. The same head was not sufficient, because retaining it while ablating the other 1{,}599 heads also produced 0\% accuracy. Two Mistral-backbone models (LLaVA-Mistral-7B and Idefics2-8B) had no critical head. An early-layer dependence was present in every model but varied in strength, whereas dependence on any single head ranged from dominant to absent. Within Qwen2.5-VL, a dominant head was present in the 7B model but not the 3B model, indicating that this concentration emerged with scale rather than being fixed across a model family, and it was not explained by attention type. These results show that ablating the head whose removal disrupts naming does not establish that the head computes the behavior, that the result generalizes across models, or that the behavior localizes to a head at all.